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ABSTRACT. This study reports the development and validation of an instrument to assess 
institutional or university-level environment in universities. Using a sample of 489 academics from 
52 departments in 28 Australian universities, an instrument called the University-Level 
Environment Questionnaire (ULEQ) was field-tested and validated. The final form of the ULEQ 
consists of 42 items assigned to seven scales: Academic Freedom, Concern for Undergraduate 
Learning, Concern for Research and Scholarship, Empowerment, Affiliation, Mission Consensus, 
and Work Pressure. Validation data showed that the ULEQ has sound structural characteristics, 
thus suggesting that it should prove to be an important research tool for individual academics, 
departments, and universities interested in improving their learning environments. 
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Introduction 



Over the paist 30 years, much learning envirorunent research has been conducted in primary and 
secondary schools. Various reviews of research attest to the extensive development of this research 
field (e.g. Fraser, 1994, 1998a). An important focus of this research has been the development and 
validation of instruments to assess particular learning environments. Although some of this research 
has investigated the environment in tutorial and workshop settings of universities (Fraser and 
Treagust, 1986), no recent studies have considered the overall or institutional environment as 
distinct from the classroom environment in universities. That is, studies of university-level 
environment which involve academic staff in reporting their perceptions of the overall climate or 
^{jj^Qgpjjere ofihe university has been a neglect^ area of research77Vpossiblc”obstaclc“to"such 
research is the lack of a contemporary instrument to assess the salient dimensions of university- 
level environment. 

The purpose of the present paper is to address this deficiency by reporting the development and 
validation of an instrument to assess university-level environment in Australian universities. Apart 
from the need to develop such an instrument, this study was important because the use of quality 
assurance mechanisms in universities in Australia and other Western countries is becoming 
accepted practice. Accordingly, university administrators should be concerned about the overall 
environment of the university. An assessment of the university-level environment should be 
regarded as one component of any broad evaluation of a university. Following this introductory 
section, background information on Australian universities, analogous school environment research 
and prior university-level environment research are discussed. The next section describes the 
instrument’s development and the structure of its final form. Comprehensive validation data for the 
instrument obtained from the sample of 489 academic staff in Australian universities are reported in 
the next main section. The concluding section suggests possible research directions using this new 
instrument. 



Background 



Australian Universities 



As in many Western countries, higher education in Australia has undergone major changes during 
the past decade. According to the Department of Employment, Education, Training and Youth 
Affairs (DEETYA, 1997), dramatic changes have occurred in size, structure, funding arrangements 
and focus of higher education institutions. In 1987, the binary system which separated 19 
universities from 69 colleges of advanced education was abolished in favour of a Unified National 
System. By 1994, amalgamations and mergers had resulted in there being 36 publicly-funded 
universities in this system. During the past decade, the Australian tertiary sector has experienced 
very high growth with student numbers in undergraduate and postgraduate studies increasing from 
394,000 in 1987 to 659,000 in 1997 (DEETYA, 1997). 

Given the reorganisation and rapid expansion of the higher education sector, the Australian 
Government recognised the importance of maintaining quality in universities by producing a range 
of government-sponsored initiatives and associated documents (Baldwin, 1991; Bowden and Sachs, 
1997; Dawkins, 1988; Higher Education Council, 1992). Baldwin presented a set of measures 
aimed at enhancing the quality of university teaching and research. Subsequently, the Higher 
O Education Council proposed that all tertiary institutions develop quality assurance and management 
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procedures. The result of these initiatives was that, from 1993 to 1995, a quality assurance program 
focusing on teaching, learning, research and community service was implemented in all Australian 
universities under the direction of a Ministerial advisory body called the Committee for Quality 
Assurance in Higher Education. To ensure compliance with the demands of the quality assurance 
process, significant financial incentives were provided to participating universities. One defining 
feature of the present Australian government’s approach to quality assurance and improvement in 
higher education is “a process that, as far as practicable, is hard-edged through the application of 
quantitative performance indicators and benchmarks which are carefully selected, genuinely 
meaningful, and recognises institutional diversity” (Higher Education Council, 1997, p. 3). Wilson 
(1996), the Chair of the Committee for Quality Assurance in Higher Education summed up the 
importance of quality assurance to Australian universities in the 1 990s by asserting that the 
worldwide phenomenon of quality in higher education must be considered the theme of the decade. 

One important determinant of learning in any school or university is the quality of its learning 
environment. In its review of teaching in British universities, the Polytechnics and Colleges 
Funding Council (1990) stated that the ethos of the whole institution largely determines the 
student’s experience and that this notion of institutional ethos is an important contributing factor to 
quality in higher education institutions. In Australia, the Higher Education Council (1992) 
suggested that the learning environment is likely to have an important effect on how students learn 
and that there are many aspects of this environment that can be influenced by government and 
institutional action to the benefit of educational quality. It follows that assessments of university- 
level environments should be an integral part of quality assurance mechanisms. According to the 
Higher Education Council (1992), administrative and support staff who do not venture formally 
into classrooms mediate the effectiveness of the teaching-learning process in universities. 

Therefore, the need to go beyond the classroom and investigate the institutional or university-level 
environment becomes acute if total quality assurance is a strategic objective of the institution. There 
is little doubt that the Australian government’s interest in quality assurance has a strong focus on 
outcomes. However, quite apart from the view that positive environments are associated with 
enhanced student learning, it needs to be accepted that good working environments in universities 
are intrinsically desirable goals. 



Analogous School Environment Research 



For many years, school environment has been acknowledged as an important and vital aspect of any 
school. Documents spanning 140 years of education in Australia allude to the central role of the 
environment in fostering student cognitive and affective growth. Erickson (1981) concluded that 
the most effective schools are distinguished, not by elaborate facilities, extensively trained teachers, 
small classes, or high levels of financial support, but by outstanding social climates. These findings 
are remarkably similar to those of Rutter et al. (1979) whose study of London schools concluded 
that school processes collectively produce a unique spirit or ethos. According to Anderson (1982), a 
large number of studies conducted in the 1960s and 1970s concluded that school environment 
enhances a range of student-centred variables: cognitive and affective outcomes (e.g. Barker, 1963; 
Brookover et al., 1978; Duke and Perry, 1978); values (e.g. Vyskocil and Goens, 1979); and 
personal growth and satisfaction (e.g. Bailey, 1979; Coyne, 1975). Other studies investigated school 
environment dimensions as criterion variables. For example, rapport between staff and 
administration was found to be related positively to school environment in studies conducted by the 
New York State Department (1976) and Ellett and Walberg (1979). Flagg (1965) established links 
between school size, teacher turnover, principal characteristics and school environment. In his 
Q review of Goodlad’s (1984) A Place Called School, Glatthom (1984) suggests that most satisfying 
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and least satisfying schools were distinguished by a subtle distinction in climate and usually 
involved relationships between teachers and students, teachers and administrators, and parents and 
school personnel. The fact that such relationships are manipulable reinforces the generally held 
view that environments are dynamic rather than static and that environments can be modified. 

Historically, school environment has been associated with the field of educational administration in 
which schools are viewed as formal organisations. As such, they are similar to most social 
groupings in that they have goals, rules, roles, a hierarchy of authority, reward systems, forms of 
compliance, coordination activities and communication patterns (Thomas, 1976). One particularly 
useful conceptualisation of school environment is in terms of the school’s organisational climate 
and psychosocial characteristics (Moos, 1974). Within this approach, schools are understood solely 
in terms of the perceptions of their inhabitants in a framework of person-milieu interaction (Fraser 
and Rentoul, 1982). Accordingly, school environment has been assessed through the perceptions of 
teachers on the school staff, primarily because they can comment on staff-related issues of which 
students might not be aware. School environment instruments typically possess several internally 
consistent and independent scales, each of which has between five and eight items. Many research 
studies conducted during the 1960s and 1970s were facilitated by the Organizational Climate 
Description Questionnaire (OCDQ), an instrument pioneered by Halpin and Croft (1963) and 
which relied heavily on previous work in business organisations. The shortcomings of the OCDQ 
have been well documented (e.g. Kottkamp et al., 1987; Thomas, 1976). 

Two instruments that have been used in recent school environment research are the Work 
Environment Scale (WES; Moos, 1986) and the School-Level Environment Questionnaire (SLEQ; 
Fraser, 1994). Although not specifically designed for use in schools, the WES has been used to 
assess ten dimensions of school environment; Involvement, Peer Cohesion, Staff Support, 
Autonomy, Task Orientation, Work Pressure, Clarity, Control, Innovation, and Physical Comfort 
(Fisher and Fraser, 1991; Fraser et al., 1988). Validation data for both of these studies attest to the 
usefulness of the WES in school settings. 

The SLEQ was developed specifically for school settings and has become an important research 
tool in Australia and the United States. The latest version of the SLEQ has eight scales: Student 
Support, Affiliation, Professional Interest, Staff Freedom, Participatory Decision Making, 
Innovation, Resource Adequacy and Work Pressure. Validation data obtained from its use in 
Australian schools are available (Fraser, 1994). A study by Fisher et al. (1993) investigated 
relationships between interpersonal teacher interaction and the school environment according to the 
SLEQ. This investigation confirmed the validity of the SLEQ for use in Australian schools. A 
modified form of the SLEQ was used to assess school-level environment in a recent Australian 
study involving principal-teacher interaction and school environment (Cresswell and Fisher, 1996; 
Fisher, 1997). Some recent school environment research directions include: investigating the 
relationship between school environment and school improvement (Dellar, 1997); establishing 
beginning teachers’ perceptions of their school environment (Heroman et al., 1997); investigating 
the relationship between school environment and teacher burnout in Singaporean schools (Ball et 
al., 1995); and studying the relationship between teacher personality and interpersonal teacher 
behaviour (Fisher et al., 1998). Templeton and Johnson (1998) used the SLEQ to assess the school 
environment of an urban school to illuminate what teachers felt to be important factors in 
developing a safer school environment. 



Prior University Environment Research 



^ Significant classroom environment research has been conducted in small group settings (e.g. 
tutorials, workshops, and science laboratories) in Australian universities during the past decade 
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(Fraser, 1998a; Fraser and Treagust, 1986). The College and University Classroom Environment 
Inventory (CUCEI) was developed by Fraser and Treagust to assess seven psychosocial dimensions 
of small groups: Personalisation, Involvement, Student Cohesiveness, Satisfaction, Task 
Orientation, Innovation, and Individualisation. Cross-national environment research in imiversity 
science laboratory classes using the Science Laboratory Environment Inventory (SLEI) was 
conducted in the early 1990s (Fraser et al., 1992). A number of existing classroom environment 
scales used in secondary schools could be modified for use in university classrooms, but a detailed 
review of these instruments is outside the scope of this paper. Several reviews of classroom 
environment literature that describe these instruments are available (see e.g. Fraser, 1994, 1998a, 
1998b). 

Other studies have focused on the environment of university departments and instructional 
processes. Using an eight-scale course perceptions questiomiaire in one British university, Ramsden 
(1979) investigated links between student perceptions of department environment and their 
approaches to learning. Independent research conducted by Gaff et al. (1976) and van Rooijen 
(1986) in The Netherlands focused on the importance of department environment in European 
universities. The study by van Rooijen employed a 58-item instrument called the University 
Learning Environment Evaluation (ULEE) to assess students’ perceptions of their department’s 
learning environment and instructional processes. In a departure from the direction of much prior 
research in the USA, Dippelhofer-Stiem (1986) conceptualised a model with five hierarchically- 
ordered levels (viz. national, university, subject, courses, and the individual person). This 
longitudinal study in five universities in the Federal Republic of Germany involved the assessment 
of students’ perceptions of these five levels of environment and students’ evaluations of these 
environments. Recently, Clarke (1995) used a semi-structured, open-ended instrument called the 
Perceptions of Learning Environments Questionnaire (PLEQ) to gather student perceptual data on 
activities and behaviours in one Queensland university. 

In contrast to most of these studies, the present research focused on the university-level 
environment, a field that was pioneered in the 1950s and 1960s by Pace and his colleagues (Pace, 
1963, 1969; Pace and Stem, 1958; Stem, 1970). The College Characteristics Index (CCI), an 
instrument that assesses 30 environment dimensions of universities and colleges, was developed in 
the 1 950s, and refined to form the College and University Environment Scales (CUES) in the 
1960s. The final version of the CUES has 160 items that assess seven dimensions of environment 
considered important to universities in the 1960s (viz. Practicality, Community, Awareness, 
Propriety, Scholarship, Campus Morale and the Quality of Teaching and Faculty-Student 
Relationships). A second instrument developed in the late 1960s is the Institutional Functioning 
Inventory (IFI: Peterson et al., 1983). This 132-item instrument has 1 1 scales: Intellectual-Aesthetic 
Curriculum, Freedom, Human Diversity, Concern for Improvement in Society, Concern for 
Undergraduate Learning, Democratic Governance, Meeting Local Needs, Self-Study and Planning, 
Concern for Advancing Knowledge, Concern for Innovation, and Institutional Esprit. 

Both the CUES and IFI were important developments in the study of university learning 
environments. However, they are inappropriate for contemporary learning environment research for 
four reasons. First, some scales would not be considered salient by academics in universities today. 
The atmosphere of universities is vastly different to that experienced on many campuses in the 
1960s. Second, with 160 and 132 items, respectively, the CUES and IFI lack economy. Their 
structure is inconsistent with contemporary instruments which have a relatively small number of 
reliable scales, each with a fairly small number of items. Third, achieving minimal overlap of scales 
is an important design criterion if results are not to be confounded. Scale intercorrelations for the 
IFI are generally high, indicating considerable overlap among its 1 1 scales (Peterson et al., 1983). 
Fourth, the CUES’s Quality of Teaching and Faculty- Student Relationships scale combines 
conceptually distinct dimensions of the environment. 



Instrument Development 



The initial development of the instrument in the present study was guided by six criteria. First, the 
instrument’s structure should be consistent with the general structure of contemporary learning 
environment instruments. A review of classroom and school environment instruments indicated 
that, ideally, the new instrument should possess several internally consistent scales that have 
minimal overlap with each other. Second, the instrument must have items and scales that are 
sensitive to different environments. Third, the instrument must provide good coverage of Moos’s 
(1974) three general categories for conceptualising human environments (viz. Relationship, 

Personal Development, and System Maintenance and System Change). Fourth, the instrument 
needed to be consistent with the nature and purpose of universities. To assist with this criterion, a 
review of relevant literature on the purpose, goals and functioning of universities suggested 
important dimensions of the institutional environment that the instrument would need to assess. 
Fifth, the instrument needed to be salient to academics. To ensure salience, scales and items were 
reviewed by several academics. Sixth, the instrument should be economical in terms of the time 
needed to answer and score it. 

Using these criteria, an 80-item preliminary instrument was developed and field-tested with a 
sample of 142 academics firom the Australian Catholic University. A series of factor, item and 
reliability analyses were employed to refine the instrument to seven scales each containing six 
items. The following section describes this instrument — called the University-Level Environment 
Questionnaire (ULEQ). Comprehensive validation data from the use of this final version of the 
ULEQ in a larger study involving 489 Australian academics are provided in a later section of this 
article. 

The final version of the ULEQ consists of 42 items with six items allocated to each of seven scales. 
Academics respond to each item using a five-point Likert response format (viz. Strongly Agree = 5, 
Agree = 4, Not Sure = 3, Disagree = 2, Strongly Disagree = 1). Twelve negatively- worded items are 
reverse scored. Scale scores are obtained by aggregating the scores for the six items of each scale. 
Table I shows, for each scale, a scale description, a typical item, and the classification according to 
Moos’s schema described earlier in this paper. One noteworthy feature of the ULEQ is its wide 
coverage of Moos’s three general categories with three Personal Development, two Relationship, 
and two System Maintenance and System Change dimensions. The ULEQ’s emphasis on personal 
development contrasts with contemporary school environment instruments. Only one scale of the 
eight-scale SLEQ falls in the personal development domain. Of the ten scales of Moos’s WES, only 
two focus on personal development (Fraser, 1994). This suggests that, compared to school 
environment instruments, the ULEQ has dimensions that reflect clear differences between 
university and school environment. 



Instrument Validation 



This section describes the sample and item, factor and scale analyses used to show the sound 
stmctural characteristics of the final form of the ULEQ. It is likely that subsequent applications of 
the ULEQ could employ the department mean rather than the individual as the unit of analysis. 
Accordingly, validation data are provided for both the individual and the department mean as imits 
O of analysis. 



TABLE I 

DESCRIPTIVE INFORMATION FOR SEVEN UNIVERSITY-LEVEL ENVIRONMENT SCALES 



Scale 


Scale description 


Typical item 


Moos’s 

schema® 


Academic 

Freedom 


The extent to which staff and 
students have academic freedom. 


Staff and students may discuss 
any topic. (+) 


P 


Concern for 

Undergraduate 

Learning 


The extent to which university 
processes and teaching 
approaches emphasise a concern 
for undergraduate learning. 


Staff members are sensitive to 
the interests, needs and 
aspirations of undergraduates. 
(+) 


P 


Concern for 
Research & 
Scholarship 


The extent to which the university 
emphasises research and 
scholarship. 


Senior academics do not 
emphasise research as an 
important institutional purpose. (- 
) 


P 


Empowerment 


The extent to which academics 
are empowered and encouraged 
to be involved in decision 
making. 


My superiors deal with me in an 
authoritarian manner. (-) 


R 


Affiliation 


The extent to which academics 
can obtain assistance, advice and 
encouragement and are made to 
feel accepted by colleagues. 


I can rely on my colleagues for 
assistance if I need it. (+) 


R 


Mission 

Consensus 


The extent to which consensus 
exists within the staff with 
regards to the overarching goals 
of the university. 


Lecturers agree on the 
university’s overall goals. (+) 


S 


Work Pressure 


The extent to which work 
pressure dominates the 
environment. 


There is constant pressure on 
academics to keep working. (+) 


S 



"P: Personal Development R: Relationship S: System Maintenance and System Change 



Sample 



The validation of the final form of the ULEQ involved a sample of 489 academics drawn from 28 
of the 36 govemment-frmded Australian universities. Australia has only two private universities 
and the imiversity sample drawn for this study reflected the diversity in the population of Australian 
universities. Participants in the study include academics from large research-orientated universities, 
comprehensive universities, former institutes of technology which have a developing research 
focus, and former colleges of education which have a tradition of undergraduate teaching. In each 
university, a total of 20 academics was drawn randomly from three types of departments that reflect 
the diversity within universities (viz. Science, Education, and English). The overall response rate 
was 87%. 



Item Analyses 



An important item characteristic is that the item is sensitive to the different environments in each 
^ department. That is, each item should describe a characteristic that is neither too rare nor too 
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common. This requirement was explored in two ways. First, the endorsement percentage (i.e. the 
percentages of the sample choosing each response) was calculated for each item. As shown in Table 
II, the responses for most items reflect a good spread of responses across the five possible 
alternatives. 

TABLE II 

ITEM ANALYSES FOR THE UNIVERSITY-LEVEL ENVIRONMENT QUESTIONNAIRE 



Item Endorsement percentages Department sensitivity analysis'* 



Number® 


Strongly 

Disagree 


Disagree 


Not sure 


Agree 


Strongly 

Agree 


Mean of % in 
agreement in each 
department 


Standard deviation 
of % in agreement 
in each department 


1 


1.0 


5.0 


6.2 


52.2 


35.6 


92.72 


9.75 


2 


14.2 


39.8 


11.2 


29.8 


5.0 


52.72 


23.16 


3 


4.9 


. 20.3 


17.9 


36.1 


20.8 


70.01 


27.23 


4 


5.2 


11.6 


8.7 


49.1 


25.4 


79.54 


22.68 


5 


4.8 


12.6 


7.9 


52.2 


22.5 


73.96 


24.02 


6 


16.9 


28.7 


27.0 


24.5 


2.9 


28.02 


24.30 


7 


0.4 


2.7 


4.7 


27.3 


64.9 


94.78 


7.56 


8 


19.3 


33.1 


35.0 


6.8 


5.8 


62.50 


27.72 


9 


4.8 


26.3 


16.5 


40.2 


12.2 


44.58 


30.34 


10 


18.0 


40.0 


8.6 


24.2 


9.2 


39.57 


30.28 


11 


33.5 


50.4 


2.1 


9.5 


4.5 


87.12 


18.90 


12 


0.6 


6.4 


9.7 


54.5 


28.8 


86.23 


15.95 


13 


17.9 


47.6 


21.2 


12.2 


1.1 


11.51 


13.66 


14 


0.8 


2.7 


4.1 


37.7 


54.7 


94.58 


8.70 


11 


2.3 


14.2 


30.6 


36.8 


16.1 


17.22 


18.41 


16 


6.6 


32.8 


22.9 


31.5 


6.2 


38.33 


26.43 


17 


3.5 


19.0 


13.6 


43.9 


20.0 


68.99 


21.72 


18 


9.1 


16.7 


4.8 


54.3 


15.1 


73.99 


20.96 


19 


34.2 


50.3 


8.7 


6.2 


0.6 


83.97 


21.06 


20 


12.4 


29.3 


28.7 


25.4 


4.2 


28.19 


20.58 


21 


57.1 


33.7 


3.9 


3.4 


1.9 


93.39 


9.80 


22 


6.9 


16.6 


33.7 


33.9 


8.9 


43.06 


28.49 


23 


1.5 


7.1 


15.9 


60.8 


14.7 


74.53 


23.00 


24 


15.7 


30.3 


9.9 


33.8 


10.3 


56.72 


31.49 


25 


5.6 


19.9 


11.8 


49.9 


12.8 


70.55 


25.38 


26 


1.9 


8.3 


9.9 


59.4 


20.5 


77.66 


25.62 


27 


2.3 


9.8 


49.1 


35.1 


3.7 


43.89 


26.47 


28 


0.8 


18.2 


6.8 


45.9 


28.3 


78.98 


20.62 


29 


2.3 


14.9 


23.0 


54.5 


5.3 


62.84 


23.48 


30 


1.3 


2.1 


5.9 


59.4 


31.3 


89.49 


17.37 


31 


36.6 


43.8 


5.3 


9.9 


4.4 


82.39 


25.65 


32 


4.5 


24.5 


11.9 


39.3 


19.8 


33.69 


23.75 


33 


1.2 


7.7 


14.9 


66.7 


9.5 


77.44 


19.16 


34 


6.2 


26.2 


44.0 


22.3 


1.3 


25.62 


25.41 


35 


0.0 


7.3 


5.4 


44.0 


43.3 


91.22 


11.60 


36 


2.7 


14.5 


14.9 


52.8 


15.1 


72.94 


20.80 


37 


12.3 


32.9 


20.5 


30.4 


3.9 


33.83 


22.10 


38 


4.5 


11.2 


5.3 


46.5 


32.5 


87.80 


19.04 


39 


25.1 


46.6 


7.5 


13.8 


7.0 


74.90 


24.28 


40 


2.7 


18.3 


9.6 


52.8 


16.6 


68.88 


24.35 


41 


6.8 


21.0 


36.7 


31.8 


3.7 


34.50 


23.68 


42 


13.0 


40.8 


10.9 


32.8 


2.5 


55.96 


25.39 



“Underlined items are reverse scored. 

‘’Statistics for department sensitivity analysis are calculated from the percentage of respondents in each department who responded 
Agree or Strongly Agree for positively- worded items or Strongly Disagree or Strongly Disagree for negatively worded items. 
Sample: 489 academic staff in 52 university departments. 
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However, these endorsement proportions do not preserve department membership. Accordingly, a 
second analysis investigating department sensitivity similar to that advocated by Pace (1969) was 
conducted in the following way. For each positively worded item and for each department, the 
percentage of respondents who responded Agree or Strongly Agree was determined. Next, the mean 
and standard deviation of this distribution were calculated. For negatively worded items, a similar 
calculation was performed using Disagree or Strongly Disagree responses. Table II shows these 
statistics. To ensure that an item is assessing a department characteristic that is neither too common 
nor too rare, the mean should be above 10 and below 90. To ensure a reasonable spread across the 
departments, a standard deviation of 10 or more is recommended by Pace. The data of Table II 
generally fit within these parameters. 



Factor Analyses 



To verify the seven-factor structure to this instrument, a principal components analysis (with 
varimax rotation) using the individual as the unit of analysis was performed. This analysis extracted 
seven factors that accounted for 52.2% of the variance. As shown in Table III, all items had 
loadings of at least 0.40 with their a priori scale and less than 0.40 with other scales. Twenty-eight 
items had loadings above 0.60 with their a priori scale. As the ratio of cases to items was 13:1, this 
seven-factor solution was considered a valid representation of the overall structure of the 
instrument. A similar analysis using the department mean as the xmit of analysis revealed a similar 
factor structure and that these seven factors accounted for 68.5% of the variance. 



Internal Consistency Reliability 



Estimates of the internal consistency of the seven scales were calculated using Cronbach’s 
coefficient alpha. As shown in Table IV, these coefficients ranged from 0.65 for the Concern for 
Research and Scholarship scale to 0.87 for the Affiliation scale with the individual as the unit of 
analysis. When the department mean was used as the unit of analysis, reliabilities ranged from 0.75 
to 0.91 for these scales. Item-scale correlations confirmed that all items had been assigned to the 
appropriate scale and that each item made an appreciable contribution to that scale’s internal 
consistency. Table IV also shows scale means and standard deviations using the individual and 
department mean as units of analysis. 

Standard error data of Table IV ranged from 0.62 for Work Pressure with the department mean as 
the unit of analysis to 2.54 for the Concern for Research and Scholarship scale with the individual 
as the unit of analysis. Thus, it is unlikely that the ‘true’ means of any of the 52 departments vary 
much from their respective obtained means. Even with the Concern for Research and Scholarship 
scale, which had the lowest internal consistency reliability (0.75), the obtained department mean 
can be expected to fall within 2.60 of the ‘true’ department mean about 95% of the time, assuming 
a normal distribution of the mean scores. 
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TABLE III 

FACTOR ANALYSIS RESULTS FOR SEVEN-FACTOR UNIVERSITY-LEVEL ENVIRONMENT 

QUESTIONNAIRE 



Item 


Academic 


Concern 


Concern for 


Empower- 


Affiliation Mission Work 


number 


freedom 


for under- 


research & 


ment 


consensus pressure 






graduate 

learning 


scholarship 






1 


.64 










8 


.67 










15 


.40 










22 


.62 










29 


.52 










36 


.63 










2 




.63 








9 




.66 








16 




.76 








23 




.69 








30 




.46 








37 




.41 








3 






.66 






10 






.42 






17 






.40 






24 






.57 






31 






.65 






38 






.61 






4 








.67 




11 








.50 




18 








.69 




25 








.71 




32 








.59 




39 








.58 




5 










.71 


12 










.70 


19 










.67 


26 










.70 


33 










.66 


40 










.60 


6 










.65 


13 










.67 


20 










.68 


27 










.50 


34 










.64 


41 










.66 


7 










.66 


14 










.42 


21 










.41 


28 










.57 


35 










.65 


42 










.61 


Factor loadings smaller that .40 have been omitted. Sample: 489 academic staff in 52 university departments. 
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TABLE IV 

INTERNAL CONSISTENCY RELIABILITY (ALPHA COEFFICIENT), SCALE MEAN, STANDARD DEVIATION AND 
STANDARD ERROR FOR SEVEN UNIVERSITY ENVIRONMENT SCALES FOR TWO UNITS OF ANALYSIS 
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Discriminant Validity 



Table V reports discriminant validity data using the mean correlation of a scale with the remaining 
six scales as a convenient index. These data indicate that the scales do overlap but not to an extent 
that would violate the ULEQ’s psychometric properties. Additionally, the conceptual 
distinctiveness of each of the scales warrants their retention in the ULEQ. 



Ability to Discriminate Between Departments 



One-way ANOVAs for department environment scales with the academic as the unit of analysis 
and department membership as the main effect showed that each scale differentiated between 
departments (p < 0.05). An estimate of the proportion of the variance in ULEQ scale scores 
attributable to department membership is provided by the eta^ statistic, which is the ratio of 
‘between’ to ‘total’ siuns of squares (Cohen and Cohen, 1975). Table V shows that eta^ values 
ranged from 15% for the Work Pressure scale to 48% for the Concern for Research and Scholarship 
scale. 



TABLE V 

MEAN CORRELATIONS AND ANOVA RESULTS FOR DEPARTMENT MEMBERSfflP DIFFERENCES IN 
PERCEPTIONS OF THE UNIVERSITY ENVIRONMENT 



Scale 


Mean Correlation 


ANOVA Results 




Individual 


Dept. Mean 


F(5 1,435) 


2 

Eta 


Academic Freedom 


.33 


.40 


2.95** 


.29 


Concern for Undergraduate 
Learning 


.23 


.30 


1.72* 


.19 


Concern for Research & 
Scholarship 


.24 


.40 


6.80** 


.48 


Empowerment 


.38 


.46 


2.31** 


.24 


Affiliation 


.38 


.51 


1.50* 


.17 


Mission Consensus 


.30 


.41 


1.75* 


.18 


Work Pressure 


.05 


.11 


1.38* 


.15 



*p < 0.05 

♦*p< 0.001 

Sample: 489 academic staff in 52 university departments. 



Conclusion 



This article has reported the development of a new instrument to assess seven dimensions of the 
institutional environment of universities (Academic Freedom, Concern for Undergraduate Learning, 
Concern for Research and Scholarship, Empowerment, Affiliation, Mission Consensus, and Work 
Q Pressure). Comprehensive validation information for this instrument, the University-Level 

ERJC 



11 



Environment Questionnaire (ULEQ), attests to its sound psychometric characteristics (e.g. factorial 
validity and internal consistency reliability) for use with academics in Australian universities. 

Three broad research directions involving the ULEQ are suggested. First, the ULEQ could be used 
by universities to monitor their institutional environment. Senior administrators should consider the 
usefulness of the ULEQ as one component in the evaluation of the overall quality of their 
universities. Second, individual university academics, departments or schools could use the ULEQ 
as a diagnostic tool. Such practical applications typically involve an initial administration of the 
questionnaire, formulation of intervention strategies based on questionnaire results, use of 
intervention strategies, and re-administration of the questionnaire to establish the effectiveness of 
the intervention (Yarrow et al., 1997). Third, the ULEQ could be employed in studies analogous to 
school environment research conducted in primary and secondary schools. These studies could 
focus on the effects of university-level environment on student outcomes and academics’ teaching 
quality and research productivity, and possible determinants of institutional enviroiunent including 
the gender composition of academic staff, governance, and university and department 
characteristics (e.g. location, type, size). 

Conceptually, the ULEQ was developed within the strong tradition of national and international 
learning environment research that has been firmly established over the past 30 years (Fraser, 
1998a). Because its scales are relevant to any university, the ULEQ or context-specific derivatives 
of the ULEQ could be used in other countries with a high degree of confidence. Clearly, studies that 
provide cross-validation data from other countries are needed to verify its generalisability. As 
quality assurance in universities is a worldwide phenomenon, it is hoped that the ULEQ will 
provide the impetus for further studies on the institutional environment in colleges and universities. 
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Appendix 



University-Level Environment Questionnaire (ULEQ) 



Instructions 

This questionnaire is about the quality of your university learning environment. Your opinion about each question is 
sought. You are asked to give your opinion on statements about your university. Please answer by circling the number 
under Strongly Agree, Agree, Not sure. Disagree or Strongly Disagree. 







Strongly 

Disagree 


Disagree 


Not 

Sure 


Agree 


Strongly 

Agree 


1. 


Students are free to express their beliefs about 
academic work. 


1 


2 


3 


4 


5 


2. 


There is not much contact between lecturers 
and students outside the classroom. 


1 


2 


3 


4 


5 


3. 


Most stalf members have had papers or books 
published in the past two or three years. 


1 


2 


3 


4 


5 


4. 


I am authorised to make decisions. 


1 


2 


3 


4 


5 


5. 


I receive encouragement from colleagues. 


1 


2 


3 


4 


5 


6. 


The organisation of this university reflects its 
goals. 


1 


2 


3 


4 


5 


7. 


There is a constant pressure on academics to 
keep working. 


1 


2 


3 


4 


5 


8. 


Controversial figures in public life are not 
allowed to address students. 


1 


2 


3 


4 


5 


9. 


Lecturers get to know most students in their 
undergraduate classes quite well. 


1 


2 


3 


4 


5 


10. 


In my faculty, there are a number of academic 
staff whose appointments primarily entail 
research rather than teaching. 


1 


2 


3 


4 


5 


11 


1 have to refer even the smallest matters to a 
senior member of staff. 


1 


2 


3 


4 


5 


12. 


1 feel accepted by other academics. 


1 


2 


3 


4 


5 


13. 


Lecturers regularly refer to the mission of the 
university when addressing university issues. 


1 


2 


3 


4 


5 


14. 


Lecturers have to work long hours to complete 
their work. 


1 


2 


3 


4 


5 


li. 


Eccentric convictions and unpopular beliefs 
among staff members and students are not liked 
by senior administrators. 


1 


2 


3 


4 


5 


16. 


Most staff members do not spend much time 
talking with students about students’ personal 
interests and concerns. 


1 


2 


3 


4 


5 


17. 


Staff promotions are based on scholarly 
publications. 


1 


2 


3 


4 


5 


18. 


1 am asked to participate in decisions 
concerning administrative policies and 
procedures. 


1 


2 


3 


4 


5 


19. 


I am ignored by other academics. 


1 


2 


3 


4 


5 


20 


The operation of this university is consistent 
with its goals. 


1 


2 


3 


4 


5 
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Continued 





Strongly Disagree 
Disagree 


Not 

Sure 


Agree 


Strongly 

Agree 


2i. 


Academics don’t have to work hard in this 1 

university. 


2 


3 


4 


5 


22. 


Staff members are free to express radical 1 

political beliefs in their classrooms. 


2 


3 


4 


5 


23. 


Staff members are sensitive to the interests, 1 

needs and aspirations of undergraduates. 


2 


3 


4 


5 


24. 


Few staff members have national or 1 

international reputations for their scholarly 
contributions. 


2 


3 


4 


5 


25. 


I am encouraged to make decisions without 1 

reference to a senior members of staff. 


2 


3 


4 


5 


26. 


I can rely on my colleagues for assistance if I 1 

need it. 


2 


3 


4 


5 


27. 


My views of the overall mission of this 1 

university are very similar to other staff 

members. 


2 


3 


4 


5 


28. 


There is no time for academics to relax. 1 


2 


3 


4 


5 


29. 


Students are encouraged to criticise academic 1 

work. 


2 


3 


4 


5 


30. 


Lecturers are interested in helping students 1 

learn. 


2 


3 


4 


5 


31- 


Senior academics do not emphasise research as 1 

an important institutional purpose. 


2 


3 


4 


5 


32. 


1 have little say in the running of the university. 1 


2 


3 


4 


5 


33. 


My colleagues take notice of my professional 1 

views and opinions. 


2 


3 


4 


5 


34. 


Lecturers agree on the university’s overall 1 

goals. 


2 


3 


4 


5 


35. 


It is hard for academics to keep up with their 1 

work load. 


2 


3 


4 


5 


36. 


Staff and students may discuss any topic. 1 


2 


3 


4 


5 


37. 


In recruiting new staff members, as much 1 

importance is given to teaching ability as to 
scholarly contribution. 


2 


3 


4 


5 


38. 


Departments hold seminars in which staff or 1 

visiting scholars discuss research. 


2 


3 


4 


5 


39. 


My superiors deal with me in an authoritarian 1 

manner. 


2 


3 


4 


5 


40. 


I have many friends among my colleagues at 1 

this university. 


2 


3 


4 


5 


41. 


The university mission statement and its 1 

associated goals are understood by academics. 


2 


3 


4 


5 


42. 


Lecturers have sufficient time to discuss 1 

academic issues informally with their 

colleagues. 


2 


3 


4 


5 



® Underlined items are reverse scored 
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